Effective monitoring of slow suspicious activites on computer networks

Slow and suspicious malicious activities on modern computer networks are increasingly hard to detect. An attacker may take days, weeks or months to complete an attack life cycle. A particular challenge is to monitor for stealthy attempts deliberately designed to stay beneath detection thresholds. This doctoral research presents a theoretical framework for effective monitoring of such activities. The main contribution of this work is a scalable monitoring scheme proposed in a Bayesian framework, which allows for detection of multiple attackers by setting a threshold using the Grubbs’ test. Second contribution is a tracing algorithm for such attacks. Network paths from a victim to its immediate visible hops are mapped and profiled in a Bayesian framework and the highest scored path is prioritised for monitoring. Third contribution explores an approach to minimise data collection by employing traffic sampling. The traffic is sampled using the stratification sampling technique with optimum allocation method. Using a 10% sampling rate was sufficient to detect simulated attackers, and some network parameters affected on sampling error. Final contribution is a target-centric monitoring scheme to detect nodes under attack. Target-centric approach is quicker to detect stealthy attacks and has potential to detect collusion as it completely independent from source information. Experiments are carried out in a simulated environment using the network simulator NS3. Anomalous traffic is generated along with normal traffic within and between networks using a Poisson arrival model. Our work addresses a key problem of network security monitoring: a scalable monitoring scheme for slow and suspicious activities. State size, in terms of a node score, is a small multiple of number of nodes in the network and hence storage is feasible for very large scale networks. Effective monitoring of slow suspicious activities on computer networks Harsha Kumara Kalutarage Digital Security & Forensics (SaFe) Research Group Department of Computing, Faculty of Engineering & Computing Coventry University, UK A thesis submitted in partial fulfilment of the requirements for the Degree of Doctor of Philosophy in Cyber Security Date: November 15, 2013